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EXECUTIVE SUMMARY 


The Knowledge Is Power Program (KIPP) is the largest network of public charter schools in the 
United States. KIPP began as a network of urban middle schools designed to serve 
underserved communities, with the goal of closing achievement gaps and preparing 
disadvantaged students to succeed in college. Prior research has demonstrated that KIPP has 
large positive impacts on student achievement (Angrist et al. 2010; Tuttle et al. 2013; Gleason 
et al. 2014; Tuttle et al. 2015; Knechtel et al. 2017). However, until now it has remained an open 
question whether these initial achievement gains (measured by improvements in standardized 
test scores) will ultimately lead to improvements in students’ longer-term outcomes, such as 
college enrollment and graduation. 


In this report, we present the results of a long-term tracking study that follows 1,177 students 
who applied to enter 1 of 13 oversubscribed KIPP middle schools through a 5th or 6th grade 
admissions lottery in 2008 or 2009. Those students are now old enough to have attended 
college for at least two years. This study uses a randomized controlled trial design to ensure 
that students who were offered admission to a KIPP middle school (the treatment group) are 
similar on average to students who did not receive an offer of admission (the control group) on 
both observable characteristics, such as prior test scores, and unobservable characteristics, 
such as levels of motivation and parental support. The study focuses on two primary research 
questions: 


1. What impact do KIPP middle schools have on students’ enrollment in a four-year college? 
2. What impact do KIPP middle schools have on persistence in four-year college programs 
during the first two years after high school graduation? 


To collect information about these outcomes, we gathered data from the National Student 
Clearinghouse on college enrollment and estimated KIPP’s impacts by comparing students in 
the treatment group with students in the control group. 


Our primary impact estimates compare students who received an admissions offer through the 
lottery to students who did not receive an admissions offer at the time of the lottery. These 
“intention to treat” impact estimates use a conservative approach that includes students in the 
treatment group even if they declined to enroll in a KIPP school after receiving an admissions 
offer. On average, students who received an admissions offer to a KIPP middle school were 6.9 
percentage points more likely to enroll in a four-year college than students who applied to KIPP 
but were not offered admission (p-value = 0.047). 


In an exploratory analysis, we adjusted this impact estimate by accounting for which students 
actually attended a KIPP school (this is often referred to as a “treatment-on-the-treated” impact 
estimate). While this involves making additional assumptions as part of the analysis, the 
resulting impact estimate can more directly measure the potential effects of attending a KIPP 
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school. After adjusting for which students attended a KIPP school following the lottery, the 
impact estimate almost doubles in size: attending KIPP following a middle school lottery 
produced an increase of 12.9 percentage points (p-value = 0.042) in enrollment rates in four- 


year college programs. In our sample, 51.8 percent of students who attended KIPP enrolled in a 


four-year college within two years after high school graduation, compared to 39.0 percent of 
control students (Figure ES.1). As we discuss in the report, a more conservative approach to 
adjusting for KIPP attendance patterns following the lottery would still produce a statistically 
significant impact estimate that is greater than 10 percentage points. An effect of this size 
represents a meaningful change in college enrollment rates. For example, the national gap in 


college enrollment rates in 2017 between white students and black or Hispanic students for any 


college type was approximately 14 percentage points among 20- and 21-year-olds (U.S. 
Census Bureau 2018). In other words, the impact of attending a KIPP school (10 to 13 
percentage points) would be almost large enough to erase the nationwide racial disparity in 
college enrollment rates. 


Figure ES.1. Impact of KIPP middle schools on four-year college enrollment 
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Note: Study includes 1,177 students who applied to enter KIPP through a middle schools via admissions lotteries 
and compares the outcomes of students offered admission to KIPP (treatment group) to those not offered 
admission (control group) at the time of the lottery. Exploratory estimates of the impact of KIPP attendance 
use the lottery as an instrument for whether a student ever attended a KIPP middle or high school. The 
model pools all 13 lottery schools, controls for baseline covariates, and includes weights to account for 
probability of assignment to the treatment or control group. 


p.p. = percentage points; ITT = intent-to-treat; TOT = treatment-on-the-treated 
“Impact estimate is significantly different from zero at the .05 level, two-tailed test. 
“Impact estimate is significantly different from zero at the .01 level, two-tailed test. 
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At this point in time, for the students in this study it is only possible to observe college 
persistence patterns for a period of two years following high school graduation. The study’s 
results for measures of early college persistence (Figure ES.2) are more ambiguous than the 
findings on initial college enrollment. In this sample, our primary impact estimates reveal that 
students who received an admissions offer to KIPP were 4.8 percentage points more likely to 
enroll immediately in a four-year college and remain enrolled for two years, but this difference 
was not statistically significant (p-value = 0.135). 


Figure ES.2. Impact on persistence through four semesters of a four-year college 
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Note: Study includes 1,177 students who applied to enter KIPP through a middle schools via admissions lotteries, 
and compares the outcomes of students offered admission to KIPP (treatment group) to those not offered 
admission (control group) at the time of the lottery. Exploratory estimates of the impact of KIPP attendance 
use the lottery as an instrument for whether a student ever attended a KIPP middle or high school. The 
model pools all 13 lottery schools, controls for baseline covariates, and includes weights to account for 
probability of assignment to the treatment or control group. 


p.p. = percentage points; ITT = intent-to-treat; TOT = treatment-on-the-treated 
“Impact estimate is significantly different from zero at the .05 level, two-tailed test. 
“Impact estimate is significantly different from zero at the .01 level, two-tailed test. 


After adjusting for which students attended KIPP after the lottery in our exploratory analysis, we 
find that 33.2 percent of students who attended KIPP enrolled immediately in a four-year college 
program after high school and persisted in college for four consecutive semesters, compared to 
24.2 percent of students who did not attend KIPP. While rates of entering and persisting in 
college were higher in the treatment group, the difference between the two groups was not large 
enough to be statistically significant (p-value = 0.129). 
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For the students in this study, it is too early to know whether KIPP will ultimately produce 
improvements in college graduation rates. Within two years of high school graduation, we have 
observed that the initial effects of KIPP middle schools on college enrollment rates are 
promising. More time will be needed to gain a clearer picture of KIPP’s effects on college 
persistence and ultimately college graduation. 
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I. INTRODUCTION 


The Knowledge Is Power Program (KIPP) is the nation’s largest network of public charter 
schools, serving more than 100,000 students across a network of more than 240 schools located 
throughout the United States. KIPP schools largely enroll low-income students of color: 
approximately 88 percent of KIPP students qualify for free or reduced-price school meals and 95 
percent are African American or Hispanic. KIPP’s model focuses on preparing these students for 
success in college, with an approach that includes emphasizing high expectations, expanding the 
school day and year, developing students’ character strengths, and empowering effective 
teachers and school leaders. 


Prior research has shown that KIPP schools have large positive impacts on student achievement, 
as measured by standardized test scores (Angrist et al. 2010; Tuttle et al. 2013; Gleason et al. 
2014; Tuttle et al. 2015; Knechtel et al. 2017). For example, Tuttle et al. (2013) used a random 
assignment design to estimate the impacts of 13 oversubscribed KIPP middle schools on student 
achievement. The design used the schools’ admission lotteries to identify a treatment group of 
students who received a lottery-based admissions offer and compared their outcomes to an 
equivalent control group of students who did not receive a lottery-based admissions offer. For 
students who attended KIPP, the study found that these KIPP middle schools produced a large 
and statistically significant gain in math achievement of 0.36 standard deviations after two years 
and a gain in English Language Arts (ELA) achievement of 0.15 standard deviations, which was 
positive but not statistically significant. That study concluded when these students were still in 
middle school, but now enough time has elapsed to observe whether these students entered 
college and are on track to complete college degrees. 


The Tuttle et al. (2013) study also paired the lottery study with a nonexperimental analysis and 
verified that a matched comparison group design produced estimates of KIPP’s achievement 
impacts that were similar to the lottery-based estimates for schools where both approaches were 
possible. Further, the study used the matched comparison group approach to examine the impacts 
of a broader sample of 41 KIPP middle schools and found that KIPP’s achievement impacts were 
positive and statistically significant in both reading and math throughout four years of middle 
school. These middle schools also had positive and statistically significant impacts on test scores 
in the subjects of science and social studies. 


Other studies have also provided evidence that KIPP schools have positive impacts for 
elementary and high school students. A lottery-based study of KIPP elementary schools found 
that an admissions offer to KIPP resulted in positive and statistically significant impacts on three 
of four measures of students’ reading and mathematics skills after three years (Tuttle et al. 2015). 
Additional evidence from that study suggested that KIPP prekindergarten programs positively 
affected student achievement above and beyond the effect of KIPP elementary schools (Knechtel 
et al. 2017). There is also evidence of positive effects in KIPP high schools. In a matched 
comparison group analysis, Tuttle et al. (2015) found that KIPP high schools had a positive and 
statistically significant impact on new KIPP entrants (students who did not transition from a 
KIPP middle school) in the subjects of math, ELA, and science (but not social studies). The 
study also found that KIPP high schools have positive effects on several aspects of college 
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preparation, including exposure to guidance counseling about college, applying to college, and 
taking Advanced Placement courses and exams. 


While the evidence of KIPP’s effectiveness in improving student test scores is widely known, 
less is known about the network’s impacts on longer-term outcomes, such as entry into and 
success in college. These are important questions given that success in college is a central pillar 
of KIPP’s mission. There are also questions about whether the success of charter school 
networks like KIPP in improving student test scores can translate into success in improving 
longer-term college outcomes (Greene 2016). Doubts about this have been bolstered by the 
mixed results from other recent lottery-based studies of charter schools’ long-term effects. One 
study found that a set of charter high schools in Boston had large positive impacts on test scores 
and on enrollment in four-year college programs (Angrist et al. 2016). Another study found 
similar positive impacts on college enrollment of a Chicago charter high school (Davis and 
Heller 2019). However, a national study of charter middle schools found that there was no 
relationship between the schools’ effects on middle school test scores and the schools’ effects on 
college enrollment (Place and Gleason 2019). In other words, the schools in that study that were 
successful in improving middle school test scores did not improve students’ postsecondary 
outcomes as well. 


Another reason to examine whether KIPP’s success in improving middle school test scores 
translates to improvements in long-term outcomes is that prior lottery-based studies did not find 
a clear pattern of effects on students’ academic attitudes that could be related to long-term 
academic success. As measured by student surveys, KIPP middle schools had no statistically 
significant effect on such outcomes as student self-control, school engagement, educational 
aspirations, or academic motivation (Tuttle et al. 2013, 2015). These survey results are somewhat 
ambiguous because it is possible that self-reported outcomes are influenced by respondents’ 
frame of reference (the standard by which they judge themselves), and this reference point could 
be different for students attending a KIPP school compared to students attending a traditional 
public school. By measuring longer-term educational outcomes, we can also assess whether early 
measures of academic attitudes and perceptions are predictive of longer-term academic success. 


By tracking students’ progression from KIPP middle school lotteries into college, this study 
provides reliable evidence on the efficacy of KIPP middle schools in improving postsecondary 
outcomes. Results from this analysis, and potential future studies tracking this sample of students 
further into their college years, will enrich our understanding of the full effects of KIPP schools 
on student outcomes beyond their test scores and inform efforts to replicate key elements of the 
KIPP model in other charter schools and traditional public schools. 


More specifically, this report presents the results of a long-term tracking study following 
students who applied to enter 1 of 13 KIPP middle schools through an admissions lottery in 2008 
or 2009 and who are now old enough to have entered college. Applying a rigorous random 
assignment design, this study provides the first rigorous estimates of the long-term effects of 
KIPP middle schools on their students, including college enrollment and early patterns of 
persistence in college degree programs. 
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ll. RESEARCH METHODS 


In this chapter, we describe the data and methods we used to estimate the effects of KIPP middle 
schools on students’ postsecondary outcomes. We begin by summarizing our research questions 

and estimation approach before defining our study sample, data sources, primary and secondary 

outcomes, and analytical model. 


A Research questions and overview of estimation methods 
This study focuses on two primary research questions: 
1. What is the impact of KIPP middle schools on students’ enrollment in a four-year college? 


2. What is the impact of KIPP middle schools on persistence in four-year college programs for 
at least two years? 


The study continues to follow the cohorts of students who participated in the randomized 
controlled trial of 13 KIPP middle schools described above (Tuttle et al. 2013). These 13 KIPP 
schools are listed in Table II.1. The original study examined the impacts of KIPP middle schools 
on outcomes observed in middle school (primarily standardized test scores) one and two years 
after the lottery. This follow-up study estimates the impacts of these middle schools on students’ 
entrance into college and persistence in postsecondary programs approximately 10 to 11 years 
after the lottery. 


Table Il.1. KIPP middle schools included in analytic sample 


State City KIPP school Yearopened 2008 lottery 2009 lotiery 
CA Los Angeles Academy of Opportunity 2003 X 
CA Los Angeles Los Angeles College Prep 2003 X 
CA San Lorenzo Summit Academy 2003 X X 
DC Washington DC KEY Academy 2001 X 
DC Washington DC WILL Academy 2006 X 
GA Atlanta WAYS Academy 2003 X 
GA East Point South Fulton Academy 2003 X X 
MA Lynn Academy Lynn 2004 X X 
NY New York City Academy New York 1995 X 
TX Austin Austin College Preparatory 2002 X 
TX Dallas TRUTH Academy 2003 X 
TX Houston Academy Middle 1995 X X 
TX San Antonio Aspire Academy 2003 X 


The lottery study design uses random assignment to form treatment and control groups, making 
it essentially a randomized experiment—the gold standard for estimating impacts. In a properly 
conducted random assignment study, the treatment group will be similar to the control group at 
the time of the lottery on both observable characteristics, such as prior test scores, and 
unobservable characteristics, such as levels of motivation and parental support. 
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B. Study sample and data collection 


The study sample consists of 1,177 students who applied to enter grade 5 or grade 6 at an 
oversubscribed KIPP middle school for either the 2008—2009 school year or the 2009-2010 
school year. The study includes data from a total of 19 admission lotteries at the 13 schools, with 
each lottery representing a distinct combination of school, cohort, and entry grade. (Appendix A 
includes a list of each school in the study). As of summer 2019, all students in the sample would 
have been old enough to have completed at least their second year of college if they followed a 
standard grade progression through middle and high school (Table II.2). 


Table II.2. Overview of possible college semester exposure by cohort 


Potential college 


Number of semesters by Percentage 
Entry grade lotteries summer 2019 Treatment (N) Control (N) Co) Tn) o) (=) 
2008 5 3 6 76 100 15 
2008 6 1 8 33 33 6 
2009 5 6 4 276 230 43 
2009 6 9 6 150 279 36 
Total 19 535 642 


Note: Since we do not directly observe high school graduation in our study sample, we assume a standard grade 
progression to estimate a student’s potential number of college semesters. 


For a KIPP school to be eligible for the study, it had to (1) be oversubscribed—have more 
applicants than open seats—for 5th or 6th grade by its scheduled lottery date, (2) conduct a 
lottery to randomly select students for admissions offers and produce a randomly ordered waitlist 
of students not selected for admission via the lottery, (3) make subsequent offers of admission to 
fill additional open seats following the randomly ordered waitlist, and (4) not exhaust the 
randomly ordered waitlist of original lottery participants through the start of the school year. A 
member of the original study team personally attended each lottery to observe the lottery 
procedures, obtain an independently verified copy of the lottery results and waitlist, and 
document any stratification used. 


The original study team excluded students from the sample if they were automatically admitted 
to the school without participating in the lottery—typically those who had a sibling already 
enrolled in the school. As a result, no students in the study’s treatment or control group had any 
siblings enrolled at KIPP at the time of the lottery. Parental consent was obtained for eligible 
applicants to participate in the study prior to the schools’ admissions lotteries, which ensured that 
there was no systematic relationship between the likelihood of consent for a given student and 
whether he or she was offered admission to the school (and thus was in the treatment group) or 
not offered admission (and thus was in the control group). The average consent rate among 
lottery participants was 75 percent and was statistically equivalent for treatment and control 
students (74 percent and 76 percent, respectively). 


To help confirm that the lotteries resulted in treatment and control groups with similar 
characteristics, we tested for differences between the two groups on key baseline student 
characteristics. Of the 23 baseline indicators available for the sample, the treatment and control 
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group differ by less than 0.12 standard deviations in all cases. There were no statistically 
significant differences on the study’s four baseline and prebaseline achievement measures, and 
there was a Statistically significant difference on only 2 of 19 measures of the students’ 
demographic characteristics (Table I.3). These small differences are consistent with the random 
amount of variation we would expect in a sample of this size. In our regression model to estimate 
impacts, we control for any remaining differences in baseline characteristics. 


These baseline characteristics also provide a picture of how the students in the study compare to 
broader populations of students. Most of the sample consists of students of color from low- 
income households. Among the treatment group, approximately 55 percent of the students are 
Hispanic, 38 percent are black, 83 percent are eligible for free or reduced-price lunch, 63 percent 
of families have incomes of less than $35,000 a year, 46 percent speak another language than 
English at home, and 50 percent of the mothers of students never enrolled in any type of 
postsecondary program. In terms of academic achievement, treatment group students had 
baseline test scores that were close to the average for their school district (scoring at the 51st 
percentile in reading and the 52nd percentile in math). 


Table II.3. Baseline equivalence for the analytic sample 


Number with 


Baseline characteristic Treatment Control Difference valid data 

Baseline reading score (z-score) 0.026 —0.029 0.055 610 
(0.080) 

Baseline math score (z-score) 0.048 -0.019 0.067 619 
(0.087) 

Prebaseline reading score (z-score) 0.004 —0.106 0.110 553 
(0.083) 

Prebaseline math score (z-score) —0.027 —0.057 0.030 556 
(0.089) 

Student is female 0.521 0.486 0.035 1,170 
(0.038) 

Age relative to cohort (in years) 0.047 0.008 0.039 1,074 
(0.035) 

Student is Hispanic 0.551 0.557 —0.006 1,094 
(0.026) 

Student is white 0.028 0.032 —0.004 1,094 
(0.011) 

Student is black 0.376 0.333 0.043 1,094 
(0.024) 

Student is other ethnicity 0.045 0.078 —0.033* 1,094 
(0.016) 

Student has an Individualized 0.106 0.130 0.024 981 

Education Program (0.026) 

Student received free or reduced-price 0.833 0.778 0.055 1,020 

lunch (0.029) 

Primary language at home is English 0.536 0.516 0.020 1,067 
(0.030) 

Household has only one adult 0.266 0.236 0.030 962 
(0.038) 

Family income is less than $15,000 0.191 0.202 —0.011 920 
(0.029) 
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Table II.3. (Continued) 


Number with 
Baseline characteristic Treatment exe} nh ice) Difference valid data 
Family income is $15,000 to less than 0.230 0.234 —0.004 920 
$25,000 (0.032) 
Family income is $25,000 to less than 0.213 0.200 0.013 920 
$35,000 (0.041) 
Family income is $35,000 to less than 0.219 0.188 0.031 920 
$55,000 (0.032) 
Family income is greater than $55,000 0.147 0.175 —0.028 920 
(0.036) 
Mother has less than a high school 0.203 0.267 —0.064* 963 
education (0.027) 
Mother completed high school 0.299 0.240 0.059 963 
education (0.033) 
Mother has some college education 0.203 0.255 —0.052 963 
(0.038) 
Mother has at least a college education 0.295 0.238 0.057 963 
(0.039) 


Note: Standard errors reported in parentheses. The difference between lottery winners and nonwinners is based 
on a regression of the characteristic on treatment status and site indicators for the original analytic sample of 
1,179. Our analytic sample omits two of these students without a valid birthdate for whom postsecondary 
outcomes could not be obtained. We reran baseline equivalence results for the 1,177-student sample, and 
the only change is that the difference between the share of students who receive free or reduced-priced 
lunch became significant with a p-value of 0.048 (the difference increased from 0.055 to 0.056 and the 
standard error fell from 0.029 to 0.028). The lottery nonwinner mean is unadjusted, and the lottery winner 
mean is the sum of the lottery nonwinner mean and the regression-adjusted difference between groups. 
Missing data were not imputed: sample sizes differ by row, due to variation in data availability by site. All of 
the baseline characteristics in this table are included as covariates in the study’s primary impact model. 


*Significantly different from zero at the 0.05 level, two-tailed test. 


Prior to the study, we estimated that this sample of students would provide enough statistical 
power to detect impacts of around 7 percentage points on the outcome of college enrollment. 
With this level of statistical power, the analysis may not be able to detect certain effect sizes that 
many would consider to be policy relevant, since we are limited by the sample size of lottery 
applicants from the original study. However, this sample size is sufficient to detect an effect on 
college enrollment that is similar in size to at least one prior study of high-performing charter 
high schools (Angrist et al. 2016). 


C. Data sources 


Our data source for measuring students’ college outcomes is the National Student Clearinghouse 
(NSC), which provides data on college enrollment, persistence, and degree completion at 
colleges and universities enrolling more than 97 percent of all public and private students in the 
United States (Dundar and Shapiro 2016). We requested data from the NSC on postsecondary 
enrollment patterns through the spring 2019 semester for 1,177 of the 1,179 students in the 
original randomized controlled trial study for whom we have valid birthdates. The NSC provides 
information on whether the students in our sample match a student in their database of those who 
attended a postsecondary institution. Students for whom there is a match are considered to have 
enrolled in a postsecondary institution. Students for whom there is no NSC match are defined as 
not having attended any postsecondary institution. We did not use information on college 
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graduation as the vast majority of the sample (94 percent) would not have had enough time to 
complete four years of college under a normal grade progression by spring 2019. 


It is possible that some students in our sample attended college but were not successfully 
matched in the NSC database due to data errors in names or birthdates, or colleges or students 
withholding enrollment data to the NSC (Dynarski et al. 2015). We used several approaches to 
mitigate this issue. We crosschecked multiple sources of students’ birthdates in our sample, and 
if there was a discrepancy we submitted both birthdates to the NSC. If a student had a middle 
name or a hyphenated last name, we submitted permutations of student names to the NSC to 
ensure that our request included each student’s official name in the NSC database. Finally, the 
NSC reported the number of student matches that were blocked by the school or student and thus 
not included in our analysis (and coded as not enrolling). The rates were similar in the treatment 
and control groups (3 percent for the control group and 2 percent for the treatment group). 
Appendix A provides more information on our NSC data request. 


In addition the NSC database, we downloaded administrative data from the Integrated 
Postsecondary Education Data System, which provides information on college enrollment, 
graduation, financial aid, and demographics. The KIPP Foundation also provided student rosters 
for KIPP schools for the duration of the follow-up period, so we could ascertain the number of 
years that each student in the sample (including those in the treatment and control groups) 
attended a KIPP school. For baseline data on the sample (measuring the characteristics of 
students before they applied to a KIPP middle school lottery), we used data collected for the 
original KIPP study, including lottery application records; a baseline survey of parents, which 
that included demographic and socioeconomic information; and administrative records from 
states, districts, or schools that provided baseline and prebaseline test score data for the study 
sample. 


D. Outcomes 


Our analysis focused on two postsecondary outcomes: college enrollment and college 
persistence.’ Based on when the students in this sample were expected to graduate from high 
school, it is too early to observe impacts on degree completion, so the study is limited to 
examining early persistence patterns over the first four college semesters. We define our primary 
measures as: 


e Ever enrolled in a four-year college. Student was enrolled in a four-year college within two 
years following the student’s expected high school graduation date. 


e Persisted through first four semesters in a four-year college. Student was enrolled in a four- 
year college for four consecutive semesters, following the student’s expected high school 
graduation date. 


Prior to collecting any data or performing data analyses, we preregistered the study’s choice of primary outcomes 
and analytical methods through the Open Science Framework, hosted by the Center for Open Science (Nichols- 
Barrer et al. 2019). 
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Because the KIPP network seeks to counsel students into selective postsecondary programs with 
relatively high graduation rates and mainly four-year degrees, the study’s primary analyses focus 
on enrollment and persistence in four-year postsecondary programs (rather than two-year 
programs). 


Since we do not directly observe high school graduation in the data collected for this study, we 
assume each student followed a standard grade progression from the date of their middle school 
admissions lottery through to the end of high school.” As a result, patterns of grade retention 
among students in the sample could introduce bias into our outcome measures if grade retention 
rates differ significantly between the treatment and control groups. This could happen because 
students who were held back a year in middle or high school would not have the opportunity to 
enroll full time in college in the fall after their expected high school graduation (since they will 
remain in secondary school for at least one year longer than expected). 


In our sample, we can observe grade retention rates among students who attended a KIPP school, 
using data provided by the KIPP Foundation. Across both middle and high school, 8 percent of 
students who ever enrolled in a KIPP school repeated a grade while enrolled at KIPP. We cannot 
directly observe grade retention rates outside of KIPP, and therefore we cannot directly compare 
the retention rates of the treatment and control groups. However, prior studies have shown that 
students at KIPP middle schools have tended to repeat a grade at higher rates than students at 
traditional public schools (Nichols-Barrer et al. 2016; Tuttle et al. 2013; Tuttle et al. 2015). Ifa 
larger share of treatment students than control students repeated a grade in their middle or high 
school years, this could lead to higher rates of on-time college enrollment in the control group 
even if the two groups were equally likely to proceed to college following their high school 
graduation. Our primary enrollment outcome measures any four-year college enrollment within 
two years of expected high school graduation. The two-year window makes this measure less 
susceptible to this issue, although we also estimated the effects of KIPP on a measure of on-time 
college enrollment (as a secondary outcome). 


To better understand and contextualize results from the two primary measures, we estimated 
impacts on secondary measures of college enrollment and persistence as well. This included 
examining enrollment and persistence at two-year colleges and any type of college. By 
measuring persistence at any type of college, we capture students who began in a two-year 
college and transitioned to a four-year college. Other secondary outcomes included on-time 
college enrollment and several variations of early college persistence. Finally, we conducted an 
exploratory analysis to measure if KIPP schools affected the types of colleges that students select 
using measures of college selectivity, graduation rate, and the economic makeup of the student 
body. Table II.4 lists all secondary outcome measures. 


” If the NSC data reported that a student was enrolled in college full time one year prior to their expected high 
school graduation, then we assumed that student graduated high school early and adjusted their expected high 
school graduation date so that they would count as enrolled in college when they appeared in the NSC data. If a 
student was only enrolled part time in college prior to their expected high school graduation, we considered that 
student dual-enrolled in high school and college and did not adjust their expected high school graduation date. 
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Table 11.4. Overview of secondary outcome measures 


Outcome 


alco) anteyarmeleicexe) antes) 


Definition 


Ever enrolled in college 


Ever enrolled in a two-year 
college 
Ever enrolled in a four-year 
college 
Enrolled on-time in college 


Persistence outcomes 


Persisted through first four 
semesters in a four-year 
college. 

Persisted through first four 
semesters in a two-year 
college. 

Number of consecutive 
semesters enrolled. 


Enrolled two springs after high 
school graduation. 


Percent of possible semesters 
enrolled. 


College program attributes 
College admissions rate 


College graduation rate 


Share of Pell Grant recipients 


Student enrolled in any type of two- or four-year college within two years following 
expected high school graduation. 

Student enrolled in a two-year college within two years following expected high 
school graduation. 

Student enrolled in a four-year college within two years following expected high 
school graduation. 

Student enrolled in college in the fall after their expected high school graduation 
date. We measured this outcome separately for students who enrolled in any 
college, in a four-year college, or in a two-year college. 


Student enrolled in four-year college programs for four consecutive semesters 
following expected high school graduation. 


Student enrolled in two-year college programs for four consecutive semesters 
following expected high school graduation. 


This measure counts the number of consecutive semesters a student enrolled, 
beginning the fall after their expected high school graduation and continuing for four 
semesters. If a student did not enroll in college in the first semester following high 
school graduation, they received a zero for this variable. We measured this 
outcome separately for students who persisted in any college, persisted only ina 
four-year college, or persisted only in a two-year college. 

Student enrolled in college in their fourth college semester following expected high 
school graduation. We measured this outcome separately for students who enrolled 
in any college, in a four-year college, or in a two-year college. 

Share of semesters that a student was enrolled in college following expected high 
school graduation for all available years of data. While many cohorts have only four 
possible semesters of enrollment data, some cohorts are old enough to have six 
semesters of college enrollment data, and one cohort has enough data for eight 
semesters. We measured this outcome separately for students enrolled in any 
college, in a four-year college, or in a two-year college. 


Four binary variables measuring whether a student enrolled in a college with an 
admissions rate of (1) 25 percent or lower, (2) greater than 25 percent and less 
than or equal to 50 percent, (3) greater than 50 percent or less than or equal to 75 
percent, or (4) greater than 75 percent. We used the admissions rate at the college 
in which the student was most recently enrolled. 


Four binary variables measuring whether a student enrolled in a college with a 
graduation rate falling in different ranges (for the college in which the student most 
recently enrolled). We used the same cut-points as for the college admissions 
variables described previously. 

Two binary variables measuring whether a student enrolled in a college with a low 
or high percentage of students who received Pell Grants during the 2017-2018 
school year (College Board 2019). One variable measured whether a student went 
to a college with above-average recipients of Pell Grants (school average was 
greater than 32 percent). The second variable measured whether a student went to 
a college with below-average recipients of Pell Grants (school average was less 
than or equal to 32 percent). 
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E. Analytical approach 


Intent-to-treat impacts. Our primary impact estimates compare students who received an 
admissions offer through the lottery to students who did not receive an admissions offer at the 
time of the lottery. These “intention to treat” (ITT) impact estimates use a conservative approach 
that includes students in the treatment group even if they declined to enroll in a KIPP school 
after receiving an admissions offer. We estimated the impacts of KIPP on postsecondary 
outcomes using the regression model in equation (1), which compares outcomes of treatment and 
control students while adjusting for differences in their baseline characteristics: 


(1) Vig = AHFO*T + PX HY, + Ex 


In this model, 7 and k index students and school lotteries, respectively, and y is the student-level 
outcome of interest (either binary or continuous). Tis a binary treatment status variable 
indicating whether the student was offered admission by lottery to the KIPP school to which he 
or she applied, and X is a set of control variables that capture student-level characteristics. These 
control variables include baseline and prebaseline math and reading test scores; gender; age 
(standardized by grade and year to reflect whether a student is young or old for their grade); 
race/ethnicity; household income; mother’s education; whether a household has only one adult; 
and if the student receives free or reduced-priced lunch, has an individual education plan, and 
primarily speaks English at home. Missing data in baseline control variables were imputed by 
regression as part of the prior study. For any remaining missing data, we set missing cases to a 
value of zero. The model includes missing-indicator dummy variables for all control variables 
with missing data. The model also controls for the school to which the student applied, the year, 
and the grade they were in (sometimes referred to as lottery fixed effects). We estimate a linear 
probability model for any binary outcomes, and the impact model incorporates sample weights 
that account for the fact that some students have a higher probability of being offered admission 
(either based on their inclusion in a particular lottery stratum defined by a student characteristic 
or because they have a sibling in the lottery). 


Our regression model is the same one used to estimate achievement impacts in the initial study of 
KIPP middle schools. We tested the model by successfully replicating the middle school 
achievement impacts from the earlier study using the follow-up study’s updated and merged data 
files. We also conducted a series of sensitivity analyses to check if our results were sensitive to 
our model specification. We estimated impacts using models without covariates, without 
imputation for baseline covariates, and using a logit instead of a linear probability model for 
binary outcomes. We also checked results using two alternative weighting approaches: (1) 
inverse-variance weights, which give greater weight to the lottery sites with more precise site- 
level impact estimates; and (2) equal site weights, which estimate the impact of the average KIPP 
lottery site. Our primary approach weights the KIPP lottery sites according to each site’s 
weighted sample size, so it effectively yields an estimate of the impact of KIPP on the average 
lottery participant. Results from our sensitivity analyses are included as Appendix B.1. 


We also explored whether KIPP impacts differed for key subgroups for our primary outcome 
measures, including subgroups defined by a student’s race, gender, socioeconomic status, and 
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baseline achievement level. Due to small sample sizes, the subgroup analysis is exploratory and 
discussed in Appendix B.2. 


Treatment-on-the-treated impacts. Our primary impact estimates (the intent-to-treat impacts) 
measure the effect of an offer of admission to a KIPP middle school, regardless of whether 
treatment group students offered admission actually attended KIPP (or, conversely, whether 
control group students who were not offered admission attended KIPP anyway). In our sample, 
not all treatment group students attended either a KIPP middle school or a KIPP high school, and 
some control group students did attend a KIPP school. However, a significantly larger proportion 
of treatment students attended a KIPP school (68 percent) than control students (16 percent), as 
shown in Figure II.1. In terms of years of attendance, the average treatment student attended 
KIPP schools for 3.45 years, compared to 0.9 years for the average control student. In other 
words, on average students in the treatment group attended KIPP for about 2.5 years longer than 
students in the control group. 


Figure Il.1. Exposure to KIPP schools, by group 
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Note: We estimated sample means by regressing KIPP attendance on treatment status using inverse probability 
weights. The control mean is unadjusted, and the treatment mean is the sum of the control mean and the 
regression-adjusted difference between groups. The difference for each indicator is statistically significant 
(p-value <0.01). Sample size = 1,177. 


To estimate the effect of actually attending a KIPP school, we conducted an exploratory analysis 
that uses the KIPP admissions lottery as an instrument for whether a student ever attended a 
KIPP school. This “treatment-on-the-treated” (TOT) model requires additional assumptions that 
are not needed to estimate ITT impacts. Specifically, to adjust for the patterns of KIPP 
attendance in both the treatment group and the control group, the analysis assumes that the 
impact of attending KIPP experienced by students in the treatment group is the same on average 
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as the impact of attending KIPP experienced by students in the control group. Our estimating 
equation follows the model used in the original KIPP study. We used two-stage least squares to 
first estimate the effect of winning an admissions lottery on KIPP attendance (IV equation 1), 
and in the second stage estimated the impact of KIPP attendance (as predicted by the lottery) on 
student outcomes (IV equation 2). 


(IV equation 1) attendKIPP,; =u+p*T+o%* Xi + VE + Six 
(IV equation 2) Vix = a+6*attendKIPP;, + B* Xin +VK + Eix 


In IV equation 2, 6 represents the impact of attending a KIPP middle or high school at any point 
after the initial school lottery. To explore whether the results from this model are robust, in the 
following chapter of the report we also discuss an alternative TOT adjustment that accounts only 
for the pattern of KIPP attendance in the treatment group, ignoring the pattern of KIPP 
attendance in the control group. In an additional analysis, we also estimated the impact of each 
year of attendance at a KIPP school, based on the model in Angrist et al. (2010) that uses the 
admissions lottery as an instrument for the number of years a student attended a KIPP school. In 
that model, 6 provides an estimate of the impact of attending a KIPP school for a single year. 
Results from that model are included in Appendix B.4. 
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lll. KIPP’S IMPACT ON COLLEGE ENROLLMENT AND PERSISTENCE 


In this chapter, we begin by describing the primary results of the study: the estimated impacts of 
KIPP middle schools on enrollment and persistence in four-year college degree programs. We 
then discuss findings from our secondary measures in those domains. Finally, we describe the 
types of colleges that students in our sample are attending and assess whether KIPP is affecting 
the type of college programs students select. 


A. Primary impacts of KIPP on enrollment and persistence in four-year 
colleges 


KIPP middle schools had a positive and statistically significant impact on enrollment in four- 
year colleges. On average, students who received a lottery-based admissions offer to a KIPP 
middle school were 6.9 percentage points more likely to enroll in a four-year college than 
students who applied to KIPP but were not offered admission (p-value 0.047). In particular, 47.8 
percent of treatment students enrolled in a four-year college within two years of high school 
graduation, compared with 41.0 percent of control students (Figure III.1). 


Figure Ill.1. Impact of KIPP middle schools on four-year college enrollment 
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Note: Study includes 1,177 students who applied to enter KIPP through a middle schools via admissions lotteries, 
and compares the outcomes of students offered admission to KIPP (treatment group) to those not offered 
admission (control group) at the time of the lottery. Exploratory estimates of the impact of KIPP attendance 
use the lottery as an instrument for whether a student ever attended a KIPP middle or high school. The 
model pools all 13 lottery schools, controls for baseline covariates, and includes weights to account for 
probability of assignment to the treatment or control group. 

p.p. = percentage points; ITT = intent-to-treat; TOT = treatment-on-the-treated 

“Impact estimate is significantly different from zero at the .05 level, two-tailed test. 


“Impact estimate is significantly different from zero at the .01 level, two-tailed test. 
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In our exploratory analysis estimating impacts among students who ended up attending a KIPP 
school, we find that the impact estimate is almost twice the size of the impact estimate based on 
admission offers alone.’ The impact of attending a KIPP school on enrolling in a four-year 
college is a statistically significant 12.9 percentage points, and represents an increase from 39.0 
percent in the control group to 51.8 percent in the treatment group. An effect of this size 
represents a meaningful change in college enrollment rates. In 2017, the national gap in college 
enrollment rates between white students and black or Hispanic students for any college type was 
approximately 14 percentage points among 20- and 21-year-olds (U.S. Census Bureau 2018). In 
other words, the impact of attending a KIPP school (12.9 percentage points) would be almost 
large enough to erase the nationwide racial disparity in college enrollment rates. 


Figure IIl.2. Impact on persistence through four semesters of a four-year college 
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Note: Study includes 1,177 students who applied to enter KIPP through a middle schools via admissions lotteries, 
and compares the outcomes of students offered admission to KIPP (treatment group) to those not offered 
admission (control group) at the time of the lottery. Exploratory estimates of the impact of KIPP attendance 
use the lottery as an instrument for whether a student ever attended a KIPP middle or high school. The 
model pools all 13 lottery schools, controls for baseline covariates, and includes weights to account for 
probability of assignment to the treatment or control group. 


p.p. = percentage points; ITT = intent-to-treat; TOT = treatment-on-the-treated 
“Impact estimate is significantly different from zero at the .05 level, two-tailed test. 
“Impact estimate is significantly different from zero at the .01 level, two-tailed test. 


3 Our treatment-on-the-treated model adjusts for the rate of KIPP attendance in both the treatment group (68%) and 
the control group (16%). Implicitly, this model assumes that attending KIPP has the same impact on college 
outcomes for the students from the treatment group and control group who attended a KIPP school. An alternative, 
outlined in Bloom (2006), avoids this assumption by adjusting only for rates of KIPP attendance in the treatment 
group, and ignoring attendance patterns in the control group. Under this alternative model, the estimated impact of 
attending KIPP would be 10.1 percentage points for enrollment in four-year colleges (p-value of 0.047). 
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In addition to enrolling in college, we also measured whether KIPP middle schools had an 
impact on students entering and persisting over time in four-year college programs (Figure III.2). 
Here we define persistence as enrollment in a four-year college beginning in the fall after high 
school graduation and continuing without interruption for two years (or four semesters). We find 
that an offer of admission to a KIPP middle school does not have a statistically significant effect 
on early college persistence: 30.4 percent of students who were offered admission to a KIPP 
middle school ended up entering and persisting in a four-year college through their first four 
semesters, compared to 25.6 percent of students not offered admission to a KIPP middle school. 
However, the difference of 4.8 percentage points is not significantly different from zero (p-value 
0.135). 


In our exploratory analysis accounting for which students ended up attending a KIPP school, 
there is a large difference between the treatment and control group but it remains statistically 
insignificant. We estimated that 33.2 percent of the students who attended KIPP were enrolled in 
four consecutive semesters at a four-year college following high school graduation, compared to 
24.2 percent of students who did not attend KIPP, although the 9 percentage point difference is 
not statistically significant (p-value 0.129). 


The magnitude of the impact estimate for four-year college enrollment (12.9 percentage points) 
is larger than the impact estimate for our measure of college persistence (9.0 percentage points). 
This could be due to multiple factors. The initial enrollment effect could be slightly fading out 
over time if students in the treatment group drop out of four-year programs at a higher rate than 
students in the control group. However, it is also possible that the difference in magnitude is 
driven by treatment group students enrolling in college later or transferring from two-year 
programs to four-year programs at higher rates relative to the control group. Both of these 
patterns would prevent a student from persisting for a full four semesters in a four-year college 
during the two years following expected high school graduation. To fully disentangle these 
effects, we will need to track these students for a longer period of time and ultimately estimate 
impacts on college graduation. 


We summarize the study’s primary intent-to-treat and exploratory treatment-on-the-treated 
impact estimates for college enrollment and persistence in Table III.1, with standard errors for 
each impact estimate reported in parentheses. We examined whether our findings on four-year 
college enrollment and early persistence were sensitive to the specifications of our impact model. 
The impact estimates remained consistent when using alternative estimation models, different 
approaches to dealing to with missing baseline data, and alternative sample weights. Results 
from our sensitivity analyses can be found in Appendix B.1. 


We also investigated whether the impact of KIPP middle schools differed for particular 
subgroups of students who may be disadvantaged, such as students with lower family income or 
lower baseline academic achievement. While subgroup sample sizes were small, we found no 
evidence that the long-term effects of KIPP middle schools differed for groups of students based 
on race, gender, income, mother’s education, or baseline test scores. Appendix B.2 provides the 
complete subgroup results. 
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Table Ill.1. Primary impact estimates for college enrollment and persistence 


Impact of KIPP middle school Impact of attending a KIPP school 

admissions offer (primary results) (exploratory results) 

Mean Mean Impact Mean Mean Impact 
ed aliat-lavmelel ceed iil: (treatment) (control) estimate p-value | (treatment) (control) estimate p-value 
Ever enrolled in four-year | 47.8% 41.0% 6.9p.p.* 0.047 51.8% 39.0% 12.9p.p.* 0.042 
college (3.4 p.p.) (6.3 p.p.) 
Persisted through first four | 30.4% 25.6% 4.8 p.p. 0.135 33.2% 24.2% 9.0 p.p. 0.129 
semesters (four-year (3.2 p.p.) (5.9 p.p.) 
college) 


Note: Standard errors are reported in parentheses under each impact estimate. Study includes 1,177 students 
who applied to enter KIPP through a middle schools via admissions lotteries, and compares the outcomes of 
students offered admission to KIPP (treatment group) to those not offered admission (control group) at the 
time of the lottery. Estimates of the impact of KIPP attendance use the lottery as an instrument for whether a 
student ever attended a KIPP middle or high school. The model pools all 13 lottery schools, controls for 
baseline covariates, and includes weights to account for probability of assignment to the treatment or control 


group. 
p.p. = percentage points 

“Impact estimate is significantly different from zero at the .05 level, two-tailed test. 
**Impact estimate is significantly different from zero at the .01 level, two-tailed test. 


B. Impacts on secondary measures of enrollment and persistence 


We now turn to examine secondary measures of college enrollment and persistence. In addition 
to looking at whether a student ever enrolled in a four-year college, we estimate the impact of 
KIPP on students enrolling in any college and in two-year colleges. We also measure impacts on 
measures of on-time college enrollment by school type. We do not find evidence that being 
offered admission to a KIPP middle school affects these other measures of college enrollment. 
The estimated impacts on these secondary outcomes are positive but not statistically significant 
(Table III.2). 


For example, 65.0 percent of the treatment group and 59.560 percent of the control group 
enrolled in any type of college within two years of high school graduation, but the difference is 
not statistically significant (p-value = 0.130). In the case of on-time college enrollment, the 
estimated impact is again not significant at the 5 percent level (p-value 0.069), although the 
magnitude of the estimated impact is similar to the estimated impact on our primary enrollment 
outcome: 6.2 percentage points for on-time enrollment and 6.9 percentage points for any 
enrollment. 


We also find no significant impacts of KIPP on measures of enrollment in two-year college 
programs (typically community college or associates degree programs). The rate of enrollment in 
a two-year college within two years of expected high school graduation was similar among 
students offered admission to a KIPP middle school (24.1 percent) and those not offered 
admission (22.4 percent). 
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Table IIl.2. Impact estimates on secondary college enrollment measures 


Outcome Mean (treatment) IM C=FeTam Coxe) alice) )) Impact estimate p-value 

Ever enrolled 

Any college 65.0% 59.5% 5.4 p.p. 0.130 
(3.6 p.p.) 

Four-year colleges 47.8% 41.0% 6.9 p.p.” 0.047 
(3.4 p.p.) 

Two-year colleges 24.1% 22.4% 1.7 p.p. 0.572 
(3.0 p.p.) 

Any college 57.3% 50.4% 6.9 p.p. 0.055 
(3.6 p.p.) 

Four-year colleges 43.1% 36.9% 6.2 p.p. 0.069 
(3.4 p.p.) 

Two-year colleges 14.2% 13.5% 0.7 p.p. 0.768 
(2.5 p.p.) 


Note: Standard errors are reported in parentheses under each impact estimate. Study includes 1,177 students 
who applied to enter KIPP through a middle schools via admissions lotteries, and compares the outcomes of 
students offered admission to KIPP (treatment group) to those not offered admission (control group) at the 
time of the lottery. Impacts are based on a regression model that pools all lottery schools, controls for 
baseline covariates, and includes weights to account for probability of assignment to the treatment or control 


group. 
p.p. = percentage points 
*Significantly different from zero at the .05 level, two-tailed test. 
**Significantly different from zero at the .01 level, two-tailed test. 


Because we had a limited follow-up period in which to observe persistence patterns among the 
study sample, we examined persistence in multiple ways that allowed for flexibility around late- 
entry into college or noncontinuous enrollment. For one measure (the percentage of possible 
semesters that a student enrolled in college), we also included more than two years of college 
data from older cohorts that have had a chance to persist in college for up to four years. We 
estimated impacts separately for two-year and four-year colleges, as well as for overall college 
persistence, since students may shift from two-year to four-year programs. We also estimated the 
average number of consecutive semesters enrolled, the share of students enrolled in the second 
spring semester following high school graduation, and the number of semesters enrolled out of 
the number of total possible semesters a student could attend college depending on their grade 
cohort. 


Similar to the finding for our primary college persistence measure, we find a pattern of positive 
impacts of KIPP on these secondary persistence measures, but the effects are not statistically 
significant (Table III.3). For example, students offered admission to KIPP persisted continuously 
for one-fifth of a semester longer at four-year colleges (1.44 semesters versus 1.25 semesters in 
the control group), although this impact was not significantly significant (p-value 0.117). 


Since entering and persisting in college would result in students being enrolled at the end of the 
two-year follow-up period, we also examined impacts on enrollment in the second spring after 
expected high school graduation. Again, the estimated impact of KIPP was positive but not 
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statistically significant—34.8 percent of the treatment group and 29.8 percent of the control 
group was enrolled in a four-year college in that semester (p-value 0.130). 


While all previous measures are restricted to the first two years after high school graduation to 
ensure comparability across grade cohorts in our sample, our final measure—the percentage of 
possible semesters enrolled—takes advantage of all available years of data for each cohort. 
However, we find the same persistence story on this measure as well. We find no significant 
differences between the treatment and the control group. For example, students offered 
admission to a KIPP middle school enrolled in a four-year college in 37.1 percent of possible 
college semesters compared with 32.7 percent among the control group (p-value 0.142). 


Table III.3. Impact estimates on secondary college persistence measures 


Outcome Mean (treatment) Mean (control) Impact estimate p-value 

Enrolled in all four semesters 

Any college 38.2% 34.1% 4.1 p.p. 0.237 
(3.4 p.p.) 

Four-year colleges 30.4% 25.6% 4.8 p.p. 0.135 
(3.2 p.p.) 

Two-year colleges 3.3% 5.8% -2.5 p.p. 0.058 
(1.3 p.p.) 

Any college 1.87 1.70 0.17 0.179 
(0.130) 

Four-year colleges 1.44 1.25 0.20 0.117 
(0.124) 

Two-year colleges 0.33 0.39 —0.06 0.394 
(0.070) 

Any college 45.8% 43.0% 2.8 p.p. 0.437 
(3.6 p.p.) 

Four-year colleges 34.8% 29.8% 5.0 p.p. 0.130 
(3.3 p.p.) 

Two-year colleges 11.0% 13.1% -2.2 p.p. 0.319 
(2.2 p.p.) 

Any college 49.3% 45.9% 3.4 p.p. 0.267 
(3.0 p.p.) 

Four-year colleges 37.1% 32.7% 4.4 p.p. 0.142 
(3.0 p.p.) 

Two-year colleges 12.2% 13.2% -1.0 p.p. 0.559 
(1.7 p.p.) 


Note: Standard errors are reported in parentheses under each impact estimate. Study includes 1,177 students 
who applied to enter KIPP through a middle schools via admissions lotteries, and compares the outcomes of 
students offered admission to KIPP (treatment group) to those not offered admission (control group) at the 
time of the lottery. Impacts are based on a regression model that pools all lottery schools, controls for 
baseline covariates, and includes weights to account for probability of assignment to the treatment or control 


group. 
p.p. = percentage points 
*Significantly different from zero at the .05 level, two-tailed test. 
**Significantly different from zero at the .01 level, two-tailed test. 
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We also estimated the impact of attending a KIPP middle school (treatment-on-the-treated 
impacts) for each of the study’s secondary outcome measures related to college enrollment and 
initial college persistence. Those results can be found in Appendix B.3. 


C. Impacts on the types of colleges that students are attending 


Among students in the study sample who went to college, we did not find significant differences 
between the treatment and control groups in the attributes of the schools they selected (Figure 
II1.3). 


Figure Ill.3. Types of colleges attended for the treatment and control groups 
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Note: Sample size contains students who ever attended a postsecondary institution. All measures have a sample 
size of 741 except college graduation rate, where the rate was missing from colleges attended by four 
students. No differences between treatment and control group are statistically significantly at the 5 percent 
level. 


For treatment students who enrolled in college, the average school had an admissions rate of 72 
percent and a graduation rate of 45 percent, with similar rates among students in the control 
group (74 percent and 44 percent respectively). Sixty-six percent of students at schools attended 
by the treatment group were nonwhite and 39 percent received Pell Grants—almost identical to 
the same rates in the control group. The two groups were also comparable on the share of 
students at their colleges who received any federal student loans. Finally, treatment and control 
group students who enrolled in college chose a similar mix of public, private, and for-profit 
colleges. Among the treatment group, 76 percent enrolled in a public institution, 23 percent in a 
private nonprofit college, and less than 1 percent in a private for-profit colleges. 
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To more directly examine the impact of KIPP on the types of colleges students selected, we 
defined a set of binary outcomes for these attributes. We find no evidence that KIPP affected the 
type of colleges attended by students in the sample, with respect to college admissions or 
graduation rates (Table I.4). Admission to a KIPP middle school did not result in students 
being more likely to enroll in highly selective colleges or colleges with higher graduation rates. 
However, there is suggestive evidence that KIPP led students to be more likely to go to a college 
where the percentage of Pell Grant recipients exceeds the national average (p-value 0.051). 


Table IIl.4. Secondary impact estimates on college type 


Outcome Mean (treatment) Mean(control) Impact estimate p-value 


College admissions rate 


Went to college with admin rate <=25% 2.8% 1.7% 1.1 p.p. 0.250 
(1.0 p.p.) 

Went to college with admin rate >25% 10.3% 9.4% 0.9 p.p. 0.641 

and <=50% (1.8 p.p.) 

Went to college with admin rate >50% 24.4% 19.3% 5.2 p.p. 0.093 

and <=75% (3.1 p.p.) 

Went to college with admin rate >75% 29.0% 30.7% —1.7 p.p. 0.646 
(3.6 p.p.) 

fore) | (Ye l-Mele-(olr-larelamc-1(-) 

Went to college with grad rate <=25% 18.3% 17.1% 1.2 p.p. 0.636 
(2.6 p.p.) 

Went to college with grad rate >25% 21.4% 18.8% 2.7 p.p. 0.370 

and <=50% (3.0 p.p.) 

Went to college with grad rate >50% 17.4% 14.8% 2.5 p.p. 0.313 

and <=75% (2.5 p.p.) 

Went to college with grad rate >75% 9.2% 10.3% -1.1 p.p. 0.651 
(2.4 p.p.) 

Pell Grants rate 

Went to college with above average 44.2% 37.2% 7.0 p.p. 0.051 

rate of recipients (3.6 p.p.) 

Went to college with below average 22.4% 23.9% —1.6 p.p. 0.634 

rate of recipients (3.2 p.p.) 


Note: Standard errors are reported in parentheses under each impact estimate. Study includes 1,177 students 
who applied to enter KIPP through a middle schools via admissions lotteries, and compares the outcomes of 
students offered admission to KIPP (treatment group) to those not offered admission (control group) at the 
time of the lottery. Impacts are based on a regression model that pools all lottery schools, controls for 
baseline covariates, and includes weights to account for probability of assignment to the treatment or control 
group. 

p.p. = percentage points 

*Significantly different from zero at the .05 level, two-tailed test. 

“Significantly different from zero at the .01 level, two-tailed test. 
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IV. DISCUSSION 


This reports presents the first experimental findings on the long-term impacts of KIPP middle 
schools, focusing on enrollment and early persistence in college. We find that KIPP had a 
positive and statistically significant impact on enrollment in four-year colleges. Our primary 
analysis revealed that receiving an admissions offer to a KIPP middle school increased a 
student’s likelihood of enrolling in a four-year college by 6.9 percentage points. In an 
exploratory analysis that adjusted for which students actually enrolled at a KIPP middle or high 
school after receiving an admissions offer, we find that attending a KIPP school had an impact of 
12.9 percentage points on the four-year college enrollment rate. 


The magnitude of this impact falls within the range of estimates from other research on the long- 
term effects of charter schools. Place and Gleason (2019) find only a small (and not statistically 
significant) impact of 3 percentage points on four-year college enrollment for students admitted 
to a nationwide sample of 30 oversubscribed charter middle schools. For one high-performing 
charter school in New York City, by contrast, admission offers led to a statistically significant 
increase of 9 percentage points in four-year college enrollment, and attendance at the school led 
to a 16 percentage point increase in college enrollment (Dobbie and Fryer 2015). Studies of 
charter high schools, where students are much closer to the point of college entry, also show 
positive impacts on four-year college enrollment (Angrist et al. 2016; Davis and Heller 2019). 
Angrist et al. (2016) found that students who attended a set of six charter high schools in Boston 
were 18 percentage points more likely to enroll in a four-year college. Unlike other studies of the 
long-term effects of charter schools, that study also found that Boston charter high schools led to 
a substantial (11 percentage point) decrease in the two-year college enrollment rate. In other 
words, attending one of those schools led a substantial proportion of students to shift from two- 
year to four-year colleges. We find no such pattern among KIPP middle schools. 


Our findings are less clear with respect to patterns of persistence in four-year college programs. 
During the first two years after high school graduation, we do not find clear evidence that KIPP 
has an effect on college persistence. Specifically, students admitted to a KIPP middle school 
were 4.8 percentage points more likely to enter and persist in a four-year college for four 
consecutive semesters. However, this difference in persistence rates was not statistically 
significant. Moreover, the difference we did observe was driven by treatment group students 
being more likely to enroll in four-year colleges, rather than any difference in persistence among 
the subset of students who entered a four-year school. 


Despite the absence of statistically significant impacts on our measures of early college 
persistence, it is worth considering the policy relevance of the magnitudes of the estimated 
impacts from our exploratory analysis of the effects of attending a KIPP school on both 
enrollment (12.9 percentage points) and persistence through the first four semesters after high 
school graduation (9.0 percentage points). Imagine, for example, that the hypothetical impact of 
attending a KIPP school on college graduation was the same as our estimate of the impact on 
persistence—9 percentage points. An impact of this size would be substantial in relation to 
differences between groups in college graduation rates. Nationally among 25- to 29 year-olds, 44 
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percent of white Americans have a bachelor’s degree compared to only 23 percent of black 
Americans and 21 percent of Hispanic Americans—a degree completion gap of 21 and 23 
percentage points respectively (National Center for Education Statistics 2019). If a future study 
revealed that KIPP middle schools ultimately do have an effect of approximately 9 percentage 
points on college completion, that effect would be equal to more than a third of the degree- 
completion gap for black and Hispanic students. 


Given the size of these potential effects, it will be important to examine the next chapter in the 
lives of these students and assess how many of them ultimately complete college degrees. In 
addition, it would also be valuable to include a larger sample of KIPP schools and students in 
future studies to obtain a more precise set of estimates of KIPP’s effects on college outcomes. 
For the students in this study, it is too early to say whether the strong effect of KIPP middle 
schools on college enrollment will ultimately improve rates of college graduation or lead to long- 
term improvements in employment and earnings after college. We look forward to future 
tracking studies that could reveal what happens next. 
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The study sample consists of 1,177 students who applied to enter grade 5 or grade 6 at an 
oversubscribed KIPP middle school for either the 2008—2009 or 2009-2010 school year. The 
study includes data from 13 KIPP middle schools holding a total of 19 admission lotteries, with 
each lottery representing a distinct combination of school, cohort, and entry grade. Sample sizes 
differ considerably by school as some schools had more eligible seats for the lottery and/or had 
more applicants to those seats. Because of this, we present aggregated impact estimates for the 
overall sample rather than estimating school-level impacts. Table A.1 provides treatment and 
control group sample sizes for each KIPP middle school that students applied to in our study 
sample. 


Table A.1. Sample sizes by KIPP middle schools 


State City KIPP school Treatment sample size Control sample size 
CA Los Angeles Academy of Opportunity 17 66 
CA Los Angeles Los Angeles College Prep 13 96 
CA San Lorenzo Summit Academy 86 39 
DC Washington DC KEY Academy 16 17 
DC Washington DC WILL Academy 6 11 
GA Atlanta WAYS Academy 17 4 
GA East Point South Fulton Academy 172 74 
MA Lynn Academy Lynn 63 102 
NY New York City Academy New York 70 15 
TX Austin Austin College Preparatory 11 45 
TX Dallas TRUTH Academy 9 8 
TX Houston Academy Middle 46 188) 
TX San Antonio Aspire Academy 9 12 
Total 535 642 


Table A.2 provides additional information on our data request to the National Student 
Clearinghouse (NSC). In particular, we provide the postsecondary data opt-out rates reported by 
the NSC for the treatment and control groups. Students and institutions have the option of 
refusing to share records through the NSC database; these cases consist of students who were 
matched to a college according to the NSC but either they or their school opted out of allowing 
their data to be shared. In our data set, we cannot differentiate between a student who blocked 
their enrollment data from being shared and a student who did not go to college. However, we 
can identify if opt-out rates are biasing our impact estimates by examining the overall opt-out 
rates separately by the treatment and control group. We find that the rates are similar and thus 
should not be biasing our impact estimates: 1.9 percent of the treatment group and 3.1 percent of 
the control group opted-out of sharing their college enrollment data. 


We also examined match rates separately by treatment and control group for the cases when we 
included a single record request for a student, as compared to cases when we included multiple 
record requests for the same student. We used the same process for the treatment and control 
groups to determine which students warranted submitting multiple records. Specifically, when a 
student had a hyphenated name or a middle name, we submitted a batch of multiple records for 
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that student with the different name permutations in the NSC submission file. There were also a 
few students in both the treatment and control groups that listed multiple birthdates in our sample 
file. In those cases, we submitted records with each of the recorded birthdates in our data set. If 
the NSC provided any matched postsecondary records for a student with multiple submissions, 
we Classified the student as having enrolled in a postsecondary program. 


We find that the match rates were similar for the students with a single record submission and for 
students with multiple record submissions. Overall, for any college type and all years of 
available data, 66.6 percent of students with single record submissions matched to the NSC, 
compared to 67.2 percent of students with multiple submitted records (this overall match rate is 
considerably higher than the enrollment rate in four-year college programs cited in the study’s 
primary findings, because a considerable portion of the matched students only enrolled in two- 
year college programs and because some of the students in older cohorts enrolled in college after 
the study’s two-year follow-up period). The match rate is also similar between single and 
multiple record submissions when examining the results separately by treatment status (the 
differences are less than | percentage point). In other words, submitting multiple or single 
records for students did not appreciably affect the match rates used to define the study’s 
outcomes in the treatment and control group. 


Table A.2. Summary of postsecondary data requested and returned 


Full sample Treatment group Oxo} nl ice) Me [cele] o) 

Percentage of students who opted out of sharing 2.5% 1.9% 3.1% 
data with the NSC 

Match rate from submitting single records 66.6% 70.5% 63.7% 
Number of single record requests 988 423 565 
Match rate from submitting multiple records 67.2% 69.6% 64.4% 
Number of multiple record requests 127 78 49 
Overall sample size Usa 535 642 


Note: |The NSC reports a count of the number of students in the data request file who were found in NSC records 
but had opted out of sharing their data. The means presented in the first row of this table represent the raw 
proportion of the treatment or control group students in the data request who opted out and are not adjusted 
for selection probability or site. The match rates reported cover any college type and all years of available 
data. They are also raw means that are not adjusted for selection probability or site. 
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1. Sensitivity analyses 


To test our modeling decisions on model form, covariate selection, baseline imputation, and 
sample weights, we conducted a series of sensitivity analyses. 


Logit model. To estimate our primary impacts, we used an ordinary least squares (OLS) 
regression model. When estimating binary outcomes (that is, the outcome has two possibilities, 
such as enrolled in college or not enrolled in college), an OLS regression model is also known as 
a linear probability model (LPM). Different model forms can plausibly produce different results, 
SO we reestimated impacts on our primary outcomes using a logit model to test how sensitive our 
findings are to the model type. We used the same covariates as our primary impacts model. To 
compare findings between logit and LPM results, we converted our logit results to treatment and 
control means and report the marginal impact. 


We find that the logit impact estimates are almost identical to the LPM impact estimates (Table 
B.1, which also includes the estimates from our primary impact model for ease of comparison). 
For our logit results, we find a positive and statistically significant impact on four-year college 
enrollment of 6.8 percentage points (compared to 6.9 percentage points for the LPM result) and a 
positive but not significant difference of 5.0 percentage points for persisting through a student’s 
first four college semesters (compared to 4.8 percentage points for the LPM result). 


Baseline imputation. Our primary impact model uses baseline data on test scores, 
socioeconomic, and demographic characteristics as covariates. For students missing data for 
some of the covariates, we imputed values by conducting single stochastic regression imputation. 
Imputation was conducted separately by treatment status and each imputation included a 
stochastic component randomly selected from the set of all residuals in the imputation equations 
to ensure that the variance of the imputed values are the same as that of the observed values (see 
Tuttle et al. 2013 for more information on the imputation process). 


To test whether our imputation method may be driving our findings, we estimated results using 
two alternative models. First, we replaced all imputed baseline values with 0 (zero imputation 
model) but kept the model identical to our primary approach otherwise (including using 
imputation flags as covariates). Second, we estimated impacts without including any baseline 
covariates (no covariates model). These models still controlled for the school to which the 
student applied, the year, and the grade they were in. 


We find that the magnitude and directionality of the impacts are similar between our primary 
model and our alternative imputation models. There were some modest shifts in the coefficient 
estimate and/or the precision of the estimates that led to the estimated impacts on college 
enrollment to be not statistically significant without including a full set of control variables or a 
more sophisticated imputation approach. With the zero imputation model, the point estimates 
drop by about | percentage point compared to our primary model. For the no covariates model, 
we see a similar drop for our enrollment measure but no change in the magnitude of the impact 
for the persistence measure. 
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Sample weights. Our primary impact model includes sample weights, known as inverse 
probability weights, to account for the fact that not all students in the lottery have the same 
probability of being offered admission to the KIPP school (that is, being selected into the 
treatment group). Some students have a higher probability of being offered admission, either 
based on their inclusion in a particular stratum defined by a student characteristic or because they 
have a sibling in the lottery. The primary sample weights follow the original RCT study (Tuttle 
et al. 2013) and are based on the procedure used in Gleason et al. (2010). 


In the simple case, where all students interested in attending a particular KIPP school enter the 
lottery and no preferences are given for siblings or other characteristics, the sample weight for a 
given student is based upon the probability that he or she ended up in the treatment or control 
group. This probability is used in the calculation of each student’s base weight. In particular, the 
base weight assigned to treatment (or control) group members is set to the inverse of the 
probability of being selected into the treatment (or control) group. We then normalize this weight 
to account for the fact that the sample will be representative of the set of all consenting lottery 
participants at that site. We set this normalization factor such that the weights of each 
experimental group sum to one-half of the total sample size within the site. Thus, the sum of all 
students’ weights within a site will be equal to the overall sample size in that site (that is, the 
number of consenting lottery participants), with the sum of weights among treatments equal to 
that among controls. 


In sites with sibling preference rules (that is, siblings of students already enrolled in a KIPP 
school have a higher likelihood of winning the lottery), the basic approach to calculating sample 
weights is the same as in the simple case above. The difference, however, is in the calculation of 
the probability of admission. No longer can we simply use the number of students offered 
admission divided by the number of lottery participants. The exact probabilities of admission 
depend on the number of sets of siblings who participate in the lottery at the school as well as the 
number of students within each sibling set. 


To test whether our impact results are being driven by our choice of sample weights, we 
reestimated our primary impacts using two alternative weighting approaches that alter the 
relative weight of sites in the estimation—the treatment and control groups continue to 
contribute equally within site, but these normalizations affect the relative weight of different sites 
in the analysis. First, we weight school impacts equally (known as equal site weights). Second, 
we weight schools by the treatment group sample size, a version of inverse-variance weights that 
provides greater weight to the lottery sites with more precise site-level impact estimates. Within 
each site, the alternative weights still account for differences between applicants in their 
probability of being offered admission (based on factors such as whether they have siblings 
applying to the school). Besides the change in weighting approach, the model is identical to our 
primary impacts model. 


We find that the alternative weighting approaches slightly increase the magnitude of the impact 
estimates and result in a significant finding for the primary persistence measure when weighting 
schools equally. The difference between the main sample weights and the alternative ones is how 
sites are weighted relative to one another. Thus, the change in estimated impacts implies that 
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sites with larger impacts are being weighted more heavily when using the alternative 
normalizations (either weighting by school or by treatment group sample size). 


Table B.1. Impact estimates on primary outcome using alternative model specifications 


Mean Mean 

(treatment) (oLo} a te))) Impact estimate p-value 
Ever enrolled in four-year Logit 47.8% 41.0% 6.8 p.p.” 0.043 
college (3.4 p.p.) 
Persisted through first four Logit 30.4% 25.4% 5.0 p.p. 0.109 
semesters (four-year college) (3.1 p.p.) 
Ever enrolled in four-year Zero imputation 47.4% 41.4% 6.0 p.p. 0.085 
college (3.5 p.p.) 
Persisted through first four Zero imputation 29.9% 26.0% 4.0 p.p. 
semesters (four-year college) (3.3 p.p.) 0.233 
Ever enrolled in four-year No covariates 47.3% 41.5% 5.9 p.p. 
college (3.8 p.p.) 0.121 
Persisted through first four No covariates 30.3% 25.6% 4.8 p.p. 
semesters (four-year college) (3.5 p.p.) 0.173 
Ever enrolled in four-year Weight schools 46.4% 38.7% 7.7 p.p.* 0.050 
college equally (3.9 p.p.) 
Persisted through first four Weight schools 29.9% 22.9% 7.0 p.p.* 0.044 
semesters (four-year college) equally (3.5 p.p.) 
Ever enrolled in four-year Weight schools by 53.0% 44.6% 8.4 p.p.” 0.025 
college treatment group (3.7 p.p.) 

sample size 
Persisted through first four Weight schools by 33.3% 27.2% 6.0 p.p. 0.087 
semesters (four-year college) treatment group (3.5 p.p.) 
sample size 

Ever enrolled in four-year Primary impacts 47.8% 41.0% 6.9 p.p.” 0.047 
college model (3.4 p.p.) 
Persisted through first four Primary impacts 30.4% 25.6% 4.8 p.p. 0.135 
semesters (four-year college) model (3.2 p.p.) 


Note: Standard errors are reported in parentheses under each impact estimate. The study includes 1,177 students 
who applied to enter KIPP through a middle schools via admissions lotteries, and compares the outcomes of 
students offered admission to KIPP (treatment group) to those not offered admission (control group) at the 
time of the lottery. 


p.p. = percentage points 
*Significantly different from zero at the .05 level, two-tailed test. 
**Significantly different from zero at the .01 level, two-tailed test. 


2. Subgroup analyses 


We also explored whether KIPP impacts differed for key subgroups for our primary outcome 
measures. We examined subgroups defined for students with low baseline math or reading scores 
(below district mean); low household income (less than $35,000 a year); low mother’s education 
(completed high school or less); as well as for students that are male, mainly speak another 
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language than English at home, Hispanic, black, or eligible for free or reduced-price lunch. 
Given our sample size, the subgroup sizes are small and results should be interpreted with 
caution. Our overall sample size is only powered to detect college enrollment impacts of around 
7 percentage points. Most of the subgroups are less than half the size of our overall sample and 
so the impacts would need to be much larger for us to be able to reliably detect these effects as 
being statistically different from zero. 


To estimate subgroup impacts, we use the same model as our primary specification (equation | 
in Section II.E) but add in a term that represents the interaction between a subgroup indicator and 
the treatment variable. The coefficient on the interaction term represents how the estimated effect 
of KIPP on a given outcome among that subgroup differs from the effect among students who 
are not in the subgroup. A positive estimate for the interaction effect indicates that KIPP has a 
more positive effect on college enrollment or persistence among the subgroup than among other 
students. Similarly, a negative estimate indicates that KIPP has a more negative effect among the 
subgroup than among other students. Estimates that are indistinguishable from 0 imply that 
KIPP’s effect is no different for students in the subgroup than for those not in the subgroup. As 
with our primary model, we control for baseline student achievement, demographics, 
socioeconomic factors, the school to which the student applied, the year, and the grade they were 
in. We also use inverse probability weights to account for students having different probabilities 
of being offered admission to a KIPP middle school. 


Table B.2 presents our subgroup findings. For all subgroups on both primary outcomes, we do 
not find any significant differences between KIPP’s effect on the two primary outcomes among 
the subgroup and KIPP’s effect among other types of students. In other words, there is no 
evidence that KIPP’s overall positive impact on enrollment in four-year colleges is different for 
any of the subgroups we tested. The magnitudes and directionality of the subgroup interaction 
effects vary widely, but this appears to be a byproduct of “noise” in the data related to the small 
sample sizes in these subgroup analyses. These results imply that the effect of KIPP middle 
schools does not appear to be concentrated among particular subgroups of students. 


Table B.2. Impact estimates on primary outcome using alternative model specifications 


Interaction Treatment 
Outcome JU] ole] colt] e) effect p-value subgroup size 
Ever enrolled in four-year college Low baseline math scores —13.3 p.p. 0.098 108 
(8.0 p.p.) 
Persisted through first four Low baseline math scores -13.2 p.p. 0.099 108 
semesters (four-year college) (8.0 p.p.) 
Ever enrolled in four-year college Low baseline reading scores —6.5 p.p. 0.410 108 
(7.9 p.p.) 
Persisted through first four Low baseline reading scores —9.5 p.p. 0.199 108 
semesters (four-year college) (7.4 p.p.) 
Ever enrolled in four-year college | Low household income 12.1 p.p. 0.125 232 
(7.9 p.p.) 
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Table B.2. (Continued) 


STU] ole] colt] eo) 


Talccde-ed(oya) 
effect 


Mathematica 


Treatment 


p-value subgroup size 


Persisted through first four 
semesters (four-year college) 


Ever enrolled in four-year college 


Persisted through first four 
semesters (four-year college) 


Ever enrolled in four-year college 


Persisted through first four 
semesters (four-year college) 


Ever enrolled in four-year college 


Persisted through first four 
semesters (four-year college) 


Ever enrolled in four-year college 


Persisted through first four 
semesters (four-year college) 


Ever enrolled in four-year college 


Persisted through first four 
semesters (four-year college) 


Ever enrolled in four-year college 


Persisted through first four 
semesters (four-year college) 


Ever enrolled in four-year college 


Persisted through first four 
semesters (four-year college) 


Ever enrolled in four-year college 


Persisted through first four 
semesters (four-year college) 


Low household income 


Low mother’s education 


Low mother’s education 


Male 


Male 


Main language at home not English 


Main language at home not English 


Hispanic 


Hispanic 


Black 


Black 


Eligible for FRPL 


Eligible for FRPL 


Black males 


Black males 


Hispanic males 


Hispanic males 


3.7 p.p. 
(6.7 p.p.) 
10.2 p.p. 
(7.3 p.p.) 


9.2 p.p. 
(6.5 p.p.) 


6.0 p.p. 
(6.5 p.p.) 


7.8 p.p. 
(5.9 p.p.) 
13.4 p.p. 
(7.1 p.p.) 


4.0 p.p. 
(6.5 p.p.) 


4.1 p.p. 
(6.9 p.p.) 


—0.8 p.p. 
(6.5 p.p.) 
—10.3 p.p. 
(7.0 p.p.) 


-7.0 p.p. 
(6.7 p.p.) 


—2.6 p.p. 
(8.1 p.p.) 


—0.3 p.p. 
(7.0 p.p.) 


—6.5 p.p. 
(7.0 p.p.) 


—5.8 p.p. 
(6.4 p.p.) 


8.4 p.p. 
(7.5 p.p.) 


8.8 p.p. 
(6.7 p.p.) 


0.584 


0.163 


0.155 


0.361 


0.184 


0.061 


0.541 


0.550 


0.907 


0.139 


0.293 


0.743 


0.961 


0.354 


0.363 


0.262 


0.190 


232 


154 


154 


261 


261 


151 


151 


185 


185 


269 


269 


321 


321 


127 


127 


96 


96 


Note: 


We used nonimputed baseline data to identify each subgroup. Standard errors are reported in parentheses. 


The study compares the outcomes of students offered admission to KIPP (treatment group) to those not 
offered admission (control group) at the time of the lottery. Impacts are based on a regression model that 
pools all lottery schools, controls for baseline covariates, and includes weights to account for probability of 
assignment to the treatment or control group. Low math and reading scores = below average test scores (z- 
scores<0); FRPL = free or reduced-price lunch; Low income = less than $35,000 in household income a 
year; Low mother’s education = high school degree or lower; p.p. = percentage points 


*Significantly different from zero at the .05 level, two-tailed test. 
“Significantly different from zero at the .01 level, two-tailed test. 
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3. Impacts of attending a KIPP school for secondary outcome measures 


Table B.3 presents the exploratory treatment-on-the-treated (TOT) impact estimates for our 
secondary outcome measures (for completeness we also include our two primary outcome 
measures in the table as well). The TOT impacts use the KIPP admission lottery as an instrument 
for whether a student ever attended a KIPP school. Thus, the effect estimates represent the 
impact of attending a KIPP middle or high school, rather than the intent-to-treat (ITT) impacts 
that represent the impact of an admissions offer to a KIPP middle school. Since only 68 percent 
of treatment group students attended a KIPP school, while 16 percent of control students ended 
up attending a KIPP school, the TOT impacts increase the magnitude of the effect size compared 
to the ITT impacts. 


For secondary enrollment measures, we estimated the impact of on-time college enrollment by 
school type as well as whether a student ever enrolled in a two-year college or any type of 
college. We also examined persistence in multiple ways that allowed for flexibility around late- 
entry into college (or noncontinuous enrollment). We estimated the average number of 
consecutive semesters enrolled, the share of students enrolled in the second spring semester 
following high school graduation, and the number of semesters enrolled out of the number of 
total possible semesters a student could attend college depending on their grade cohort. 


While an admissions offer to a KIPP middle school (ITT model) did not have a statistically 
significant impact on any of our secondary enrollment and persistence measures, we did find that 
two of the outcomes became significant when measuring the impact of attending a KIPP school 
(TOT model) resulting from small changes in p-values. Under the TOT model, attending a KIPP 
school increased the likelihood of on-time college enrollment in any type of college by a 
statistically significant 13 percentage points. KIPP has a similar effect on students enrolling in 
colleges where the student body has an above average share of Pell Grant recipients. No other 
secondary outcome measure is statistically significant at the 5 percent level. 


Table B.3. Impact estimates on secondary outcome measures (treatment-on-the-treated) 


Mean Mean Impact of 


Outcome (treatment) (control) attending KIPP p-value 


=AV(=Y m=) a) ce) | (Xe | 


Any college 68.2% 57.9% 10.2 p.p. 0.119 
(6.6 p.p.) 

Four-year colleges? 51.8% 39.0% 12.9 p.p.* 0.042 
(6.3 p.p.) 

Two-year colleges 25.1% 21.9% 3.2 p.p. 0.563 
(5.5 p.p.) 

Any college 61.4% 48.3% 13.0 p.p.* 0.049 
(6.6 p.p.) 

Four-year colleges 46.7% 35.1% 11.7 p.p. 0.063 
(6.3 p.p.) 

Two-year colleges 14.6% 13.38% 1.4 p.p. 0.762 
(4.6 p.p.) 
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Table B.3. (Continued) 


Mean Mean Impact of 


Outcome (treatment) (control) attending KIPP 


Enrolled in all four semesters 


Any college 40.6% 33.0% 7.6 p.p. 0.228 
(6.3 p.p.) 

Four-year colleges? 33.2% 24.2% 9.0 p.p. 0.129 
(5.9 p.p.) 

Two-year colleges 1.9% 6.5% —4.6 p.p. 0.055 
(2.4 p.p.) 

Any college 1.98 1.65 0.33 0.169 
(0.239) 

Four-year colleges 1.56 1.19 0.37 0.110 
(0.229) 

Two-year colleges 0.30 0.41 0.11 0.385 
(0.129) 

Any college 47.4% 42.1% 5.3 p.p. 0.426 
(6.6 p.p.) 

Four-year colleges 37.7% 28.4% 9.3 p.p. 0.123 
(6.0 p.p.) 

Two-year colleges 9.7% 13.7% —4.1 p.p. 0.311 
(4.0 p.p.) 

Any college 51.3% 44.9% 6.4 p.p. 0.254 
(5.6 p.p.) 

Four-year colleges 39.7% 31.4% 8.2 p.p. 0.134 
(5.5 p.p.) 

Two-year colleges 11.6% 13.5% —1.9 p.p. 0.551 
(3.2 p.p.) 

Went to college with admin rate <=25% 3.4% 1.4% 2.1 p.p. 0.238 
(1.8 p.p.) 

Went to college with admin rate >25% and 10.8% 9.2% 1.6 p.p. 0.632 

<=50% (3.3 p.p.) 

Went to college with admin rate >50% and 27.5% 17.8% 9.7 p.p. 0.087 

<=/5% (5.7 p.p.) 

Went to college with admin rate >75% 28.0% 31.2% —3.2 p.p. 0.639 
(6.7 p.p.) 

Went to college with grad rate <=25% 19.1% 16.8% 2.3 p.p. 0.627 
(4.7 p.p.) 

Went to college with grad rate >25% and <=50% 23.0% 18.0% 5.0 p.p. 0.360 
(5.5 p.p.) 

Went to college with grad rate >50% and <=75% 18.8% 14.1% 4.7 p.p. 0.301 
(4.6 p.p.) 

Went to college with grad rate >75% 8.6% 10.6% —2.0 p.p. 0.644 
(4.3 p.p.) 
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Table B.3. (Continued) 


Mean Mean Impact of 
Outcome (treatment) (control) attending KIPP p-value 


Pell Grants rate 


Went to college with above average rate of 48.3% 35.1% 13.1 p.p.* 0.046 
recipients (6.6 p.p.) 
Went to college with below average rate of 21.5% 24.3% —2.9 p.p. 0.627 
recipients (5.9 p.p.) 


Note: Study includes 1,177 students who applied to enter KIPP middle schools via admissions lotteries, and 
compares the outcomes of students offered admission to KIPP (treatment group) to those not offered 
admission (control group) at the time of the lottery. Estimates of the impact of KIPP attendance use the 
lottery as an instrument for whether a student ever attended a KIPP middle or high school. The regression 
model pools all 13 lottery schools, controls for baseline covariates, and includes weights to account for 
probability of assignment to the treatment or control group. Standard errors are reported in parentheses 
under each impact estimate. Sample for college graduation rate variables = 1,173. 


4 Primary outcome measure 

p.p. = percentage points 

*Significantly different from zero at the .05 level, two-tailed test. 
**Significantly different from zero at the .01 level, two-tailed test. 


4. Annual impacts of KIPP attendance for primary outcome measures 


In addition to estimating postsecondary impacts for students who attended a KIPP middle school, 
we also estimated the impact of attending KIPP for a single year. This model assumes that the 
impact of attending KIPP is equal across years, and that these annual impact accumulate in a 
linear fashion across both middle and high school. To estimate these annual impacts, we used a 
model similar to the exploratory TOT model presented in the main text (IV equations 1 and 2), 
but used the KIPP admission lottery as an instrument for the number of years that a student 
attended a KIPP school. We find that attending a KIPP school for one year increases the 
likelihood that a student will enroll in a four-year college by a statistically significant 2.5 
percentage points (Table B.4). The impact estimate for college persistence through four 
semesters is 1.7 percentage points but is not statistically significant. 


Table B.4. Annual impact of KIPP attendance on college enrollment and persistence 


Impact of one-year of KIPP attendance 


ed aliat-lavmelel Ceol iil=) Impact estimate p-value Sample size 

Ever enrolled in four-year college 2.5 p.p.* 0.039 1,177 
(1.2 p.p.) 

Persisted through first four semesters (four- 1.7 p.p. 0.125 1,177 

year college) (1.1 p.p.) 


Note: Standard errors are reported in parentheses under each impact estimate. Estimates of the impact of one- 
year of KIPP attendance use the lottery as an instrument for the number of years that a student attended a 
KIPP school. The regression model pools all 13 lottery schools, controls for baseline covariates, and 
includes weights to account for probability of assignment to the treatment or control group. 


p.p. = percentage points. 
“Impact estimate is significantly different from zero at the .05 level, two-tailed test. 
“Impact estimate is significantly different from zero at the .01 level, two-tailed test. 
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